Method of examining specimens and system thereof

ABSTRACT

A system, method and computer readable medium for examining a specimen, the method comprising: obtaining defects of interest (DOIs) and false alarms (FAs) from a review subset selected from a group of potential defects received from an inspection tool, each potential defect is associated with attribute values defining a location of the potential defect in an attribute space; generating a representative subset of the group, comprising potential defects selected in accordance with a distribution of the potential defects within the attribute space, and indicating the potential defects in the representative subset as FA; and upon training a classifier using data informative of the attribute values of the DOIs, the potential defects of the representative subset, and respective indications thereof as DOIs or FAs, applying the classifier to at least some of the potential defects to obtain an estimation of a number of expected DOIs in the specimen.

TECHNICAL FIELD

The presently disclosed subject matter relates, in general, to the fieldof examination of a specimen, and more specifically, to methods andsystems for defect detection of a specimen.

BACKGROUND

Current demands for high density and performance, associated with ultralarge scale integration of fabricated devices, require submicronfeatures, increased transistor and circuit speeds, and improvedreliability. Such demands require formation of device features with highprecision and uniformity, which, in turn, necessitate careful monitoringof the fabrication process, including automated examination of thedevices while they are still in the form of semiconductor wafers. It isnoted that the fabrication process can include pre-manufacture,manufacture and/or post-manufacture operations.

The term “specimen” used in this specification should be expansivelyconstrued to cover any kind of wafer, masks, and other structures,combinations and/or parts thereof used for manufacturing semiconductorintegrated circuits, magnetic heads, flat panel displays, and othersemiconductor-fabricated articles.

The term “examination” used in this specification should be expansivelyconstrued to cover any kind of metrology-related operations as well asoperations related to detection and/or classification of defects in aspecimen during its fabrication. Examination is carried out by usingnon-destructive examination tools during or after manufacture of thespecimen to be examined. By way of non-limiting example, the examinationprocess can include runtime scanning (in a single or in multiple scans),sampling, reviewing, measuring, classifying and/or other operationsprovided with regard to the specimen or parts thereof using the same ordifferent examination tools. Likewise, at least some examination can becarried out prior to manufacture of the specimen to be examined, and caninclude, for example, generating an examination recipe(s), trainingrespective classifiers or other machine learning-related tools and/orother setup operations. It is noted that, unless specifically statedotherwise, the term “examination” or its derivatives used in thisspecification, is not limited with respect to resolution or to the sizeof an inspection area. A variety of non-destructive examination toolsincludes, by way of non-limiting example, scanning electron microscopes,atomic force microscopes, optical inspection tools, etc.

By way of non-limiting example, run-time examination can employ atwo-phase procedure, e.g. inspection of a specimen followed by review ofsampled locations of potential defects. During the first phase, thesurface of a specimen is inspected at high-speed and relativelylow-resolution. In the first phase, a defect map is produced to showlocations on the specimen suspected of having high probability of adefect. During the second phase, at least some of such suspectedlocations are more thoroughly analyzed with relatively high resolution.In some cases both phases can be implemented by the same inspectiontool, and, in some other cases, these two phases are implemented bydifferent inspection tools.

Examination processes are used at various steps during semiconductorfabrication to detect and classify defects on specimens. Examinationgenerally involves generating some output (e.g., images, signals, etc.)for a wafer by directing light or electrons to the wafer, and detectingthe light or electrons from the wafer. Once the output has beengenerated, defect detection is typically performed by applying a defectdetection method and/or algorithm to the output. Most often, the goal ofexamination is to provide high sensitivity to defects of interest, whilesuppressing detection of nuisance and noise on the wafer.

GENERAL DESCRIPTION

In accordance with certain aspects of the presently disclosed subjectmatter, there is provided a system to examine a specimen, the systemcomprising: a processing and memory circuitry (PMC) comprising aprocessor operatively coupled to a memory, the PMC configured to: obtaina plurality of defects of interest (DOIs) and a plurality of falsealarms (FAs), from a review subset selected from a group of potentialdefects received from an inspection tool, wherein each potential defectis associated with a plurality of attribute values defining a locationof the potential defect in an attribute space; generate a representativesubset of the group of potential defects, the representative subsetcomprising potential defects selected in accordance with a distributionof the group of potential defects within the attribute space, andindicate the potential defects in the representative subset as FA; andupon training a classifier using data informative of the attributevalues of the DOIs, the potential defects of the representative subset,and respective indications thereof as DOIs or FAs, apply the classifierto at least some of the potential defects to obtain an estimation of anumber of expected DOIs in the specimen.

In addition to the above features, the method according to this aspectof the presently disclosed subject matter can comprise one or more offeatures (i) to (x) listed below, in any desired combination orpermutation which is technically possible:

(i) Within the examination system, the PMC can be configured to selectthe representative subset as a coreset of the group of potentialdefects.(ii) Within the examination system, the PMC can be configured to selectthe coreset of the group of potential defects using a K means parallelalgorithm.(iii) Within the examination system, the representative subset can begenerated so as to have no overlap with the DOIs.(iv) Within the examination system, the PMC can be configured to obtainthe plurality of DOIs and the plurality of FAs as follows: select thereview subset of potential defects from the group of potential defects;for each given potential defect from the review subset of potentialdefects: obtain an indication whether a given potential defect is adefect of interest or a false alarm, the indication being based on areceived review tool image of the given defect; and respectively assignthe given potential defect to the plurality of DOIs or the plurality ofFAs.(v) Within the examination system, the PMC can be configured to selectthe review subset of potential defects, obtain the indication, andassign the given potential defect as follows: cluster the group ofpotential defects to obtain a plurality of clusters; sample an initialgroup of potential defects from each of the plurality of clusters, inaccordance with scores of the potential defects obtained using one ormore utility functions; receive review results of the potential defectsin the initial group of potential defects, and associate a label witheach defect indicating whether each defect therein is a defect ofinterest or a false alarm; determine whether a predetermined reviewbudget is exhausted; and if not, update the scores of remainingpotential defects based on the review results, and repeat sampling,receiving the review results and associating.(vi) Within the examination system, the group of potential defects canbe a cluster of potential defects, and the classifier is trained andapplied for the potential defects in the cluster.(vii) Within the examination system, the number of expected DOIs can beverified for validating the classifier.(viii) Within the examination system, the PMC can be further configuredto estimate False alarm rate (FAR) using the number of expected DOIs.(ix) Within the examination system, the estimation of a number ofexpected defects of interest in the specimen can be used to determine abudget of potential defects to be reviewed by a review tool whenexamining an additional specimen of a same type as the specimen.(x) Within the examination system, the PMC can be a part of theinspection tool, or part of a review tool, or part of a combinedinspection and review tool that is operated at different modes, orseparate from the inspection tool and from the review tool.

In accordance with other aspects of the presently disclosed subjectmatter, there is provided a method of examining a specimen, comprising:obtaining a plurality of defects of interest (DOIs) and a plurality offalse alarms (FAs), from a review subset selected from a group ofpotential defects received from an inspection tool, wherein eachpotential defect is associated with a plurality of attribute valuesdefining a location of the potential defect in an attribute space;generating a representative subset of the group of potential defects,the representative subset comprising potential defects selected inaccordance with a distribution of the group of potential defects withinthe attribute space, and indicating the potential defects in therepresentative subset as FA; and upon training a classifier using datainformative of the attribute values of the DOIs, the potential defectsof the representative subset, and respective indications thereof as DOIsor FAs, applying the classifier to at least some of the potentialdefects to obtain an estimation of a number of expected DOIs in thespecimen.

This aspect of the disclosed subject matter can comprise one or more offeatures (i) to (x) listed above with respect to the system, mutatismutandis, in any desired combination or permutation which is technicallypossible.

In accordance with other aspects of the presently disclosed subjectmatter, there is provided a non-transitory computer readable mediumcomprising instructions that, when executed by a computer, cause thecomputer to perform a method of examination of a specimen, the methodcomprising: obtaining a plurality of defects of interest (DOIs) and aplurality of false alarms (FAs), from a review subset selected from agroup of potential defects received from an inspection tool, whereineach potential defect is associated with a plurality of attribute valuesdefining a location of the potential defect in an attribute space;generating a representative subset of the group of potential defects,the representative subset comprising potential defects selected inaccordance with a distribution of the group of potential defects withinthe attribute space, and indicate the potential defects in therepresentative subset as FA; and upon training a classifier using datainformative of the attribute values of the DOIs, the potential defectsof the representative subset, and respective indications thereof as DOIsor FAs, applying the classifier to at least some of the potentialdefects to obtain an estimation of a number of expected DOIs in thespecimen.

This aspect of the disclosed subject matter can comprise one or more offeatures (i) to (x) listed above with respect to the system, mutatismutandis, in any desired combination or permutation which is technicallypossible.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to understand the invention and to see how it can be carriedout in practice, embodiments will be described, by way of non-limitingexamples, with reference to the accompanying drawings, in which:

FIG. 1 illustrates a block diagram of an examination system, inaccordance with certain embodiments of the presently disclosed subjectmatter;

FIG. 2 illustrates an exemplary flowchart of examining a specimen, inaccordance with certain embodiments of the presently disclosed subjectmatter;

FIG. 3A illustrates an exemplary flowchart of selecting, reviewing andassigning labels to potential defects, in accordance with certainembodiments of the presently disclosed subject matter;

FIG. 3B illustrates an exemplary flowchart of defect classification percluster, in accordance with certain embodiments of the presentlydisclosed subject matter; and

FIG. 4A and FIG. 4B show exemplary methods for estimating a false alarmrate of the potential defects within a cluster, in accordance withcertain embodiments of the presently disclosed subject matter.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the invention.However, it will be understood by those skilled in the art that thepresently disclosed subject matter may be practiced without thesespecific details. In other instances, well-known methods, procedures,components and circuits have not been described in detail so as not toobscure the presently disclosed subject matter.

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specificationdiscussions utilizing terms such as “obtaining”, “generating”,“receiving”, “training”, “applying”, “determining”, “selecting”,“assigning”, “clustering”, “sampling”, “associating”, “updating”,“repeating”, “validating”, “estimating”, or the like, refer to theaction(s) and/or process(es) of a computer that manipulate and/ortransform data into other data, said data represented as physical, suchas electronic, quantities and/or said data representing the physicalobjects. The term “computer” should be expansively construed to coverany kind of hardware-based electronic device with data processingcapabilities including, by way of non-limiting example, the examinationsystem and respective parts thereof disclosed in the presentapplication.

The terms “non-transitory memory” and “non-transitory storage medium”used herein should be expansively construed to cover any volatile ornon-volatile computer memory suitable to the presently disclosed subjectmatter.

The term “defect” used in this specification should be expansivelyconstrued to cover any kind of abnormality or undesirable feature formedon or within a specimen.

The term “design data” used in the specification should be expansivelyconstrued to cover any data indicative of hierarchical physical design(layout) of a specimen. Design data can be provided by a respectivedesigner and/or can be derived from the physical design (e.g. throughcomplex simulation, simple geometric and Boolean operations, etc.).Design data can be provided in different formats as, by way ofnon-limiting examples, GDSII format, OASIS format, etc. Design data canbe presented in vector format, grayscale intensity image format, orotherwise.

The term “sampling” used herein should be expansively construed to coverany selecting of one or more specimen locations from a collection ofspecimen locations obtained from an inspection tool or from any othersource, for example received from a user, extracted from design data,reported by previous processes, received from external sources, orothers. The sampled specimen locations may be selected from amongst thecollection of specimen locations, to be reviewed by a review tool. Asdetailed below, each location can be described as a collection ofattribute values, thus the collection of locations spans an attributespace. Sampling may be broadly construed as selecting a set of locationsthat represent the spanned attribute space.

It is appreciated that, unless specifically stated otherwise, certainfeatures of the presently disclosed subject matter, which are describedin the context of separate embodiments, can also be provided incombination in a single embodiment. Conversely, various features of thepresently disclosed subject matter, which are described in the contextof a single embodiment, can also be provided separately or in anysuitable sub-combination. In the following detailed description,numerous specific details are set forth in order to provide a thoroughunderstanding of the methods and apparatus.

Bearing this in mind, attention is drawn to FIG. 1 illustrating ageneralized block diagram of an examination system in accordance withcertain embodiments of the presently disclosed subject matter.Examination system 100 illustrated in FIG. 1 can be used for examinationof a specimen (e.g. of a wafer and/or parts thereof) as a part ofspecimen fabrication. Examination can be part of the specimenfabrication, and can be carried out during manufacturing the specimen,or afterwards. The examination system can comprise a variety ofexamination tools, for example, one or more inspection tools 101configured to capture inspection images (captured typically atrelatively high-speed and/or low-resolution, e.g., by an opticalinspection system, low-resolution SEM, etc.) and output potentialdefects, e.g., locations at which a defect may be found, and one or morereview tools 102 configured to capture review images of at least some ofthe potential defects detected by inspection tools 101 typically, atrelatively low-speed and/or high-resolution, e.g. by a scanning electronmicroscope (SEM) or Atomic Force Microscopy (AFM).

As mentioned above, images of a desired location on a specimen can becaptured at different resolutions. In some embodiments, images of thesame location (with the same or different resolutions) can compriseseveral images registered therebetween.

A specimen can be examined by inspection tool 101. The resulting imagesand/or derivatives can be processed (optionally together with other dataas, for example, design data and/or defect classification data) toselect potential defects for review, to assess the number of defects,and/or to characterize the defects in the specimen.

Illustrated examination system 100 comprises a computer-based automateddefect classification tool 103. Defect classification tool 103 can beoperatively connected to one or more inspection tools 101 and/or one ormore review tools 102. Optionally, defect classification tool 103 can bea standalone tool, or fully or partly integrated with or hosted by oneor more inspection tools 101 or review tools 102. Defect classificationtool 103 can be further operatively connected to design server 110and/or data repository 109.

By way of non-limiting example, defect classification tool 103 can beusable for different purposes. For example, for automaticallyclassifying potential defects provided by inspection examination tool101 into a plurality of classes, and in particular into defects ofinterest (DOIs) and false alarms (FAs); filtering FAs from DOIs,identifying specific DOIs, assessing the number of DOIs in a specimen ortheir characteristics, selecting some of the potential defects providedby inspection tool 101 for review by review tool 102, establishingPareto in order to identify excursions in statistical process control(SPC), and/or others.

Defects can be represented as a collection of attributes, wherein eachdefect is associated with a value for one or more of the attributes.Some attributes can be numeric and may be assigned any value from afinite or infinite range; other attributes may be assigned discretenumeric or non-numeric values. Thus, each defect represents a point inthe attribute space spawned by the possible attribute values. A metricmay be defined for determining the distance between two defects in theattribute space, based on their attribute values.

Defect classification tool 103 may be configured to receive, via inputinterface 105, input data. The input data can include image data (and/orderivatives thereof and/or metadata associated therewith) produced bythe inspection tool 101 and/or review tool 102 and/or data stored indesign data server 110 and/or one or more data repositories. In someembodiments, the input data can include one or more runtime images.

Defect classification tool 103 comprises a processor and memorycircuitry (PMC) 104 operatively connected to a hardware-based inputinterface 105 and to a hardware-based output interface 106. PMC 104 canbe a part of inspection tool 101, a part of review tool 102, or a partof a combined tool combining inspection tool 101 and review tool 102operated at different modes.

PMC 104 is configured to provide processing necessary for operating thedefect classification tool 103 as further detailed below, and comprisesa processor and a memory (not shown separately within PMC). Theoperation of defect classification tool 103 and PMC 104 is furtherdetailed with reference to FIGS. 2-4 below.

PMC 104 can be configured to execute several functional modules inaccordance with computer-readable instructions implemented on anon-transitory computer-readable storage medium. Such functional modulesare referred to hereinafter as comprised in PMC 104. PMC 104 cancomprise defect classification unit 115, configured to assess the numberand characteristics of DOIs within the specimen, select some of thedefects provided by inspection system tool 101 to be reviewed by reviewtool 102, identify excursions in statistical process control (SPC), orthe like.

Defect classification unit 115 can comprise clustering engine 114configured to receive a collection of points in an attribute space andcluster them into two or more clusters, such that the distance between afirst point and another point in the same cluster is smaller than thedistance between the first point and a third point assigned to anothercluster. Thus, clustering engine 114 may be used for clustering aplurality of potential defects, in accordance with one or more metricsdefined for the attribute space.

Defect classification unit 115 can comprise representative subsetgeneration engine 116 configured to select defects from a given group ofdefects. Representative subset generation engine 116 can select defectsin accordance with the distribution of the given group of defects in theattribute space.

Defect classification unit 115 can comprise training set obtainingengine 118, for gathering a training set for training a classifier. Thetraining set can comprise potential defects provided by inspectionexamination tool 101 that have been reviewed by review tool 102 andlabeled as a DOI or a FA in accordance with the review results.Additionally, the training set can comprise a subset of the potentialdefects provided by inspection examination tool 101, as selected, forexample, by representative subset generation engine 116, and labeled asFAs, as detailed below.

Defect classification unit 115 can comprise training engine 120configured to train one or more engines such as classification engine121 upon a training set including the corresponding labels, as obtainedby training set obtaining engine 118. Training engine 120 is thuscapable of receiving a training set comprising defects and labelsthereof, and determining separation planes and probability distributionplanes to be used by a classifier. The separation planes form sub-spaceswithin the attribute space, such that all defects in the same sub-spaceare associated with the same class, for example one or more FA classesor one or more DOI classes. The better the training set represents thepotential defect population, the better is the classification, since theplanes as determined upon the training set are applicable to thepotential defect population. It will be appreciated that the disclosureis equally applicable for any type of classifier, for example classicalclassifiers, deep neural networks, or others.

Defect classification unit 115 can comprise classification engine 121,as trained by training engine 120 and can be used for classifying thepotential defects provided by inspection examination tool 101. In someembodiments, classification engine 121 may be used for classifying onlydefects that have not been reviewed by review tool 102. Classificationengine 121 is capable of automatically classifying defects, for examplein accordance with the separation planes as determined during training.Classification engine 121 can be configured to define, for each givendefect, a confidence level indicative of probability that the defectbelongs to a certain class, and to assign the given defect to thecertain class if the confidence level meets a respective confidencethreshold.

Classification engine 121 may also use the classification results forany purpose, for example assessing the number of DOIs in the potentialdefects provided by inspection examination tool 101 without reviewingall of them since this is impractical, obtaining classification thereof,selecting defects to be reviewed by review tool 102, or the like.

Illustrated examination system 100 can comprise data repository 109,which can store, for example, data related to the specimen, toinspection results of the specimen, or the like.

Illustrated examination system 100 can comprise storage system 107 forstoring data related to the examination of the specimen, such asdetected DOIs, detected FAs, additional potential defects, thresholds,or the like.

Illustrated examination system 100 can comprise design server 110,comprising design data of the specimen, such as Computer Aided Design(CAD) data.

Illustrated examination system 100 can comprise GUI 108 for receivinginput from a user, such as the number of potential defects to bereviewed, and providing output to the user, such as the estimated numberin DOIs in the specimen, their characteristics, or the like. GUI 108 canbe further configured to enable user-specified inputs related tooperating system 100. Operation of system 100, PMC 104 and thefunctional modules therein will be further detailed with reference toFIGS. 2-4 below.

Those skilled in the art will readily appreciate that the teachings ofthe presently disclosed subject matter are not bound by the systemillustrated in FIG. 1; equivalent and/or modified functionality can beconsolidated or divided in another manner and can be implemented in anyappropriate combination of software with firmware and hardware.

It is noted that the examination system illustrated in FIG. 1 can beimplemented in a distributed computing environment, in which theaforementioned functional modules shown in FIG. 1 can be distributedover several local and/or remote devices, and can be linked through acommunication network. It is further noted that in other embodiments atleast some of the examination tools 101 and/or 102, data repository 109,storage system 107 and/or GUI 108 can be external to examination system100 and operate in data communication with defect classification tool103. Defect classification tool 103 can be implemented as a stand-alonecomputer(s) to be used in conjunction with one or more examinationtools. Optionally, defect classification tool 103 can operate onpre-acquired inspection data 121′ stored in data repository 109 and/orstorage system 107. Alternatively or additionally, the respectivefunctions of defect classification tool 103 can, at least partly, beintegrated with one or more examination tools, process control tools,recipe generation tools, systems for automatic defects review and/orclassification, and/or other systems related to examination.

Reference is now made to FIG. 2, showing a flowchart of a method ofexamining a specimen, in accordance with certain embodiments of thepresently disclosed subject matter.

PMC 104 can obtain (200) a plurality of potential defects, for examplefrom inspection examination tool 101, from data repository 109, or fromany other source. In a typical wafer the order of magnitude of thenumber of potential defects may be between tens of thousands andmillions, thus although it is highly important to detect DOIs, it isimpractical to review each and every potential defect in order todetermine whether it is a DOI, a FA, or a nuisance. Therefore, a smallfraction of the potential defects, for example between a few dozens anda few thousand defects, may be reviewed, and conclusions may be drawnfrom the review results.

PMC 104 can thus obtain (204) a plurality of DOIs and FAs. The DOIs andFAs can be used as part of a training set for training a classifier forobtaining further information on the specimen.

Specifically, for obtaining DOIs and FAs, PMC 104 can select (206) asubset of the potential defects to be reviewed, for example by reviewtool 102, obtain (208) DOI or FA indication for each reviewed potentialdefect, and assign (210) each reviewed defect to the DOIs or to the FAs,depending on the received indication.

Selection step (206), obtaining indications step (208) and assigningstep (210) are further detailed in association with FIG. 3A below inaccordance with certain embodiments of the presently disclosed subjectmatter.

Representative subset generation engine 116 can generate (212) arepresentative subset of the potential defects, in accordance with thedistribution of the potential defects within the attribute space. Thus,an objective of the selection can be to obtain a subset that representsthe dense areas of potential defects, but also potential defects fromsparser areas, although in smaller numbers. The representative subsetmay be selected not to include any of the reviewed potential defects.

The subset may be selected in a variety of ways. One such way is randomsampling. However, in order to closely represent the potential defectpopulation, a significant number of the potential defects need to beselected, for example 20%.

By making a smart selection, that takes into account the distribution ofthe population, a much smaller number of potential defects may besufficient. Such a smartly selected subset is referred to herein as acoreset, and one exemplified method for selecting the coreset is the Kmeans parallel algorithm. In general, the K means parallel algorithmoperates as follows: randomly selecting a first defect, then selecting asecond defect that is farthest from the first, selecting a third defectthat is farthest form the first and the second defects, and so on. Therepresentative set may be selected such that it contains no overlap withthe reviewed defects.

Due to the characteristics and nature of optical scanning, a significantmajority of the potential defects as provided by inspection examinationtool 101, for example over 95%, are FAs. Therefore, the defects of therepresentative subset are also considered as such, and therefore markedas FAs, although they have not been reviewed. It will be appreciatedthat even if some of the defects of the representative set are actuallyDOIs rather than FAs, such mistakes will not have any significant effecton the training that follows.

Training set obtaining engine 118 can collect the representative subset(such as the coreset) generated on step 212 which is defined to consistof FAs, the DOIs obtained at step 204 and optionally the FAs obtained atstep 204, thereby obtaining a training set.

Training engine 120 can then train (216) a classification engine 121,also referred to as classifier, using the training set comprised atleast of the review results, i.e., the DOIs as determined by review tool102, and the representative subset selected on step f comprisingpotential defects all defined as FAs. In some embodiments, the FAs asdetermined by review tool 102 may also be used for training.

Upon training classification engine 121, the trained classificationengine 121 can then be applied (220) to the potential defects, to obtainDOI and FA estimations. In some embodiments, classification engine 121can be applied only to potential defects that have not been reviewed.However, classification engine 121 can be applied to the wholepopulation of potential defects including also potential defects thathave been reviewed and it is thus known whether they are a DOI or a FA,in order to evaluate the accuracy of classification engine 121. In someembodiments, estimations can be provided for the potential defectswithin each area separated by the separation planes.

The results of applying classification engine 121 can be utilized (224)in a variety of ways, for example groups of potential defects associatedwith one or more DOI classes of FA classes may be determined. Theclassification results can also enable a user to conduct meaningfulstatistical process control (SCP), obtain a probability for eachpotential defect to be a DOI, estimate of the number of DOIs vs. thenumber of FAs in the potential defects, or the like. The results of theclassifier may also be used for determining the number of reviews to beperformed, i.e., the review budget, when examining further specimens ofthe type of the specimen being examined. By way of another example, thefalse alarm rate (FAR) may be estimated. The FAR calculation is detailedin association with FIG. 4A and FIG. 4B below.

In some embodiments, validation (228) of the classification results canbe performed, e.g., by a customer, such as the manufacturer of thespecimen. By way of example, the DOIs as classified by the classifier,or part thereof, can be reviewed by the customer for verifying whetheror not they are classified correctly, thereby evaluating the accuracy ofthe classifier trained on step 216 above.

According to certain embodiments, the above described defectclassification process with reference to FIG. 2 can be implementedcluster-wise, as will be detailed further with reference to FIG. 3B.

Reference is now made to FIG. 3A, showing a detailed flow chart ofselection step (206), obtaining indications step (208) and assigningstep (210), in accordance with certain embodiments of the presentlydisclosed subject matter.

Clustering engine 114 can cluster (300) the potential defects asreceived from inspection tool 101. The potential defects can beclustered into two or more clusters in the attribute space, wherein thedistance between first and second potential defects assigned to the samecluster is lower than the distance between the first or second potentialdefects and other potential defects assigned to other clusters. Thedistance between two potential defects is determined based on a metric,which can depend on the attributes and their types. Clustering can beautomatic, manual, or a combination thereof, wherein automaticclustering may be subject to human intervention. For example, a user mayprovide initial clusters, discover that a cluster should be split intotwo or more clusters, since it contains potential defects that areinherently different. A user may also discover two separate clustersthat should be unified. A user may change the attributes extracted fromthe inspection of the specimen, and/or the metric applied forcalculating the distance between potential defects, however such changeswill generally be used for future specimen examinations.

Initial subset generation engine 116 may sample (304) a subset ofpotential defects from each cluster.

Sampling step (304) may be performed in an iterative approach. Forexample, a first batch of samples can be selected from all the clustersusing one or more utility functions, each utility function assigning ascore to each potential defect, and the scores may be combined, forexample using a weighted combination.

In some embodiments, the utility functions may include at least oneunsupervised utility function and at least one supervised utilityfunction. All utility functions may perform calculations involving theattribute values of the potential defect. As detailed below, samplingstep (304) may be performed iteratively. On the first iteration, when nodata is available, only unsupervised utility functions may be used,while subsequent iterations may also involve supervised utilityfunctions, which provide further information on the cluster.

The sampled defects may then be imaged, for example using review tool102. PMC 104 may receive (308) the review results and associate a labelof DOI or FA with each reviewed potential defect.

The selected potential defects may be reviewed and PMC 104 may receive(308) the review results and associate with each reviewed potentialdefect a label of DOI or FA.

PMC 104 can then update (312) the application of the utility functionsbased on the review results. By way of example, the reviewed results canbe used to recalculate the score for each remaining potential defect(based on the supervised and unsupervised functions) and an updatedscore for each potential defect can be obtained. In one example, there-calculation of the score may relate to a distance between theremaining potential defects and the reviewed DOIs. In some cases, on thesecond iteration, a weight may be given to the supervised utilityfunctions, and on further iterations the weights of the supervisedutility functions may be increased.

PMC 104 can then determine (316) whether the predetermined reviewbudget, e.g. the number of reviews allotted for the whole population ofpotential defects, has been reached. If not, execution may return tosampling step (304) for sampling additional batches of potentialdefects, based on the updated scores. If the review budget is exhausted,the flow as described in FIG. 3A has completed and the process returnsto FIG. 2 and continues with step 212, as described above.

It will be appreciated that although the steps are presentedsequentially for each cluster before moving on to the next cluster, thisis not necessary, and execution can be started for a second cluster whena first cluster is still in progress. However, such an arrangementnecessitates monitoring of the total number of executed reviews, ratherthan the number performed for each cluster.

It will also be appreciated that the sample, review and label assignmentmethod may be directly applied to the whole potential defects populationrather than to respective clusters. In such an embodiment, clusteringstep (300) and further cluster determination step (320) may be omitted,and the iterative sampling process as described in steps 304, 308, 312and 316 can be applied to the whole potential defect population.

Reference is now made to FIG. 3B, showing a detailed flowchart of defectclassification per cluster, in accordance with certain embodiments ofthe presently disclosed subject matter.

Steps 300, 304, 308, and 312 are described above with reference to FIG.3A, and are thus not repeated here. Once the review results are received(step 308) and the utility function application is updated (step 312),PMC 104 can determine (318) whether a predetermined number of DOIs arereceived for any given cluster. If there is a given cluster for which apredetermined number of DOIs are received, training engine 120 canreceive, for the given cluster, the reviewed potential defects togetherwith the DOI and FA labels, and train (324) a corresponding classifierfor the given cluster.

In some embodiments, further potential defects may be selected for thetraining set from the given cluster, for example a coreset. The furtherpotential defects selected may be assumed to be false alarms, asdetailed in the step of generating the representative subset (212)above. In such cases, the classifier can be trained (324) using thereceived DOIs and FAs, as well as the coreset indicated as FAs.

The trained classifier may be applied (328) to the non-reviewedpotential defects of the given cluster (or alternatively to the wholepopulation in the cluster). Following the classification, the resultsper cluster can be utilized in a similar manner as described above withreference to step 224.

PMC 104 may determine (328) whether a predetermined review budget forthe whole population of potential defects is exhausted, and if so, theprocess has been completed. If not, the given cluster (for which theclassifier is trained) can be excluded (332) from the population, givingrise to the remaining clusters, which will be further sampled inaccordance with step 304.

It will be appreciated that the method of FIG. 3B may be repeated forsome or all of the clusters until the review budget is exhausted.

It will also be appreciated that the training classifier and applyingthe classifier to potential defects in the cluster as described withreference to steps 324 and 328 can be performed for a given clusterimmediately once the condition in 318 is fulfilled, or, alternatively,these steps can be performed for all clusters once the review budget isexhausted and the method of FIG. 3B is completed.

Reference is now made to FIG. 4A and FIG. 4B, showing exemplary methodsfor estimating the FAR of the potential defects within a cluster as onepossible way of utilizing the results of applying classifier to thepotential defects (step 224), in accordance with certain embodiments ofthe presently disclosed subject matter.

Reference is now made to FIG. 4A, wherein step 404 displays oneestimation method, being the Bayesian method which may include:

On step 408, the density of DOIs and FAs within the cluster may becalculated. In some embodiments, the density may be calculated using theKernel Density Estimation (KDE).

On step 412, the log likelihood of each potential defect to be a DOI (ora FA) may be calculated; and

On step 416, the following actions may be performed for each value inthe log-likelihood range (e.g. a working point):

-   -   Calculating the False Alarm Rate (FAR) based on the number of        DOIs and FAs available for the cluster;    -   Identifying the samples that comply with this value and adding        them to the projected subset; and    -   Estimating the FAR confidence level.

Reference is now made to FIG. 4B, wherein step 420 displays a secondestimation method, being the 2-Class Support Vector Machine (SVM)method, which may include:

On step 424, the log-likelihood may be calculated based on the number ofDOIs and unknown potential defect populations.

On step 428 the following actions may be performed for each value in thelog-likelihood range (e.g. a working point):

-   -   Calculating the FAR based on the number of DOIs and FAs;    -   Identifying the unknown potential defects that comply with the        value and adding them to the selected subset; and    -   Estimating the FAR confidence level.

The estimated FAR provides an indication of the probability of defectsin the particular cluster to be false alarms.

It is to be understood that the invention is not limited in itsapplication to the details set forth in the description contained hereinor illustrated in the drawings. The invention is capable of otherembodiments and of being practiced and carried out in various ways.Hence, it is to be understood that the phraseology and terminologyemployed herein are for the purpose of description and should not beregarded as limiting. As such, those skilled in the art will appreciatethat the conception upon which this disclosure is based may readily beutilized as a basis for designing other structures, methods, and systemsfor carrying out the several purposes of the presently disclosed subjectmatter.

It will also be understood that the system according to the inventionmay be, at least partly, implemented on a suitably programmed computer.Likewise, the invention contemplates a computer program being readableby a computer for executing the method of the invention. The inventionfurther contemplates a non-transitory computer-readable memory tangiblyembodying a program of instructions executable by the computer forexecuting the method of the invention.

Those skilled in the art will readily appreciate that variousmodifications and changes can be applied to the embodiments of theinvention as hereinbefore described.

1. A system to examine a specimen, the system comprising: a processingand memory circuitry (PMC) comprising a processor operatively coupled toa memory, the PMC configured to: obtain a plurality of defects ofinterest (DOIs) and a plurality of false alarms (FAs), from a reviewsubset selected from a group of potential defects received from aninspection tool, wherein each potential defect is associated with aplurality of attribute values defining a location of the potentialdefect in an attribute space; generate a representative subset of thegroup of potential defects, the representative subset comprisingpotential defects selected in accordance with a distribution of thegroup of potential defects within the attribute space, and indicate thepotential defects in the representative subset as FA; and upon traininga classifier using data informative of the attribute values of the DOIs,the potential defects of the representative subset, and respectiveindications thereof as DOIs or FAs, apply the classifier to at leastsome of the potential defects to obtain an estimation of a number ofexpected DOIs in the specimen.
 2. The examination system of claim 1,wherein the PMC is configured to select the representative subset as acoreset of the group of potential defects.
 3. The examination system ofclaim 2, wherein the PMC is configured to select the coreset of thegroup of potential defects using a K means parallel algorithm.
 4. Theexamination system of claim 1, wherein the PMC is configured to obtainthe plurality of DOIs and the plurality of FAs as follows: select thereview subset of potential defects from the group of potential defects;for each given potential defect from the review subset of potentialdefects: obtain an indication whether a given potential defect is adefect of interest or a false alarm, the indication being based on areceived review tool image of the given defect; and respectively assignthe given potential defect to the plurality of DOIs or the plurality ofFAs.
 5. The examination system of claim 4, wherein the PMC is configuredto select the review subset of potential defects, obtain the indicationand assign the given potential defect as follows: cluster the group ofpotential defects to obtain a plurality of clusters; sample an initialgroup of potential defects from each of the plurality of clusters, inaccordance with scores of the potential defects obtained using one ormore utility functions; receive review results of the potential defectsin the initial group of potential defects, and associate a label witheach defect indicating whether each defect therein is a defect ofinterest or a false alarm; determine whether a predetermined reviewbudget is exhausted; and if not, update the scores of remainingpotential defects based on the review results, and repeat said sampling,said receiving the review results and said associating.
 6. Theexamination system of claim 1, wherein the group of potential defectsare a cluster of potential defects, and the classifier is trained andapplied for the potential defects in the cluster.
 7. The examinationsystem of claim 1, wherein the number of expected DOIs are verified forvalidating the classifier.
 8. The examination system of claim 1, whereinthe PMC is further configured to estimate False alarm rate (FAR) usingthe number of expected DOIs.
 9. The examination system of claim 1,wherein the estimation of a number of expected defects of interest inthe specimen is usable to determine a budget of potential defects to bereviewed by a review tool when examining an additional specimen of asame type as the specimen.
 10. The examination system of claim 1,wherein the PMC is a part of the inspection tool, or part of a reviewtool, or part of a combined inspection and review tool that is operatedat different modes, or separate from the inspection tool and from thereview tool.
 11. A method of examining a specimen, comprising: obtaininga plurality of defects of interest (DOIs) and a plurality of falsealarms (FAs), from a review subset selected from a group of potentialdefects received from an inspection tool, wherein each potential defectis associated with a plurality of attribute values defining a locationof the potential defect in an attribute space; generating arepresentative subset of the group of potential defects, therepresentative subset comprising potential defects selected inaccordance with a distribution of the group of potential defects withinthe attribute space, and indicating the potential defects in therepresentative subset as FA; and upon training a classifier using datainformative of the attribute values of the DOIs, the potential defectsof the representative subset, and respective indications thereof as DOIsor FAs, applying the classifier to at least some of the potentialdefects to obtain an estimation of a number of expected DOIs in thespecimen.
 12. The method of claim 11, wherein the representative subsetof the group of potential defects is a coreset of the group of potentialdefects.
 13. The method of claim 12, wherein the coreset of the group ofpotential defects is selected using a K means parallel algorithm. 14.The method of claim 11, wherein the representative subset is generatedso as to have no overlap with the DOIs.
 15. The method of claim 11wherein obtaining the plurality of DOIs and the plurality of FAs isperformed as follows: selecting the review subset of potential defectsfrom the group of potential defects; for each given potential defectfrom the review subset of potential defects: obtaining an indicationwhether a given potential defect is a defect of interest or a falsealarm, the indication being based on a received review tool image of thegiven defect; and respectively assigning the given potential defect tothe plurality of DOIs or the plurality of FAs.
 16. The method of claim15, wherein selecting the review subset of potential defects, obtainingthe indication, and assigning the given potential defect is performed asfollows: clustering the group of potential defects to obtain a pluralityof clusters; sampling an initial group of potential defects from each ofthe plurality of clusters, in accordance with scores of the potentialdefects obtained using one or more utility functions; receiving reviewresults of the potential defects in the initial group of potentialdefects, and associating a label with each defect indicating whethereach defect therein is a defect of interest or a false alarm;determining whether a predetermined review budget is exhausted; and ifnot, updating the scores of remaining potential defects based on thereview results; and repeating said sampling, said receiving the reviewresults and said associating.
 17. The method of claim 11, wherein thegroup of potential defects are a cluster of potential defects, and theclassifier is trained and applied for the potential defects in thecluster.
 18. The method of claim 11, wherein the number of expected DOIsare verified for validating the classifier.
 19. The method of claim 11,wherein False alarm rate (FAR) is estimated using the number of expectedDOIs.
 20. A non-transitory computer readable medium comprisinginstructions that, when executed by a computer, cause the computer toperform a method of examination of a specimen, the method comprising:obtaining a plurality of defects of interest (DOIs) and a plurality offalse alarms (FAs), from a review subset selected from a group ofpotential defects received from an inspection tool, wherein eachpotential defect is associated with a plurality of attribute valuesdefining a location of the potential defect in an attribute space;generating a representative subset of the group of potential defects,the representative subset comprising potential defects selected inaccordance with a distribution of the group of potential defects withinthe attribute space, and indicating the potential defects in therepresentative subset as FA; and upon training a classifier using datainformative of the attribute values of the DOIs, the potential defectsof the representative subset, and respective indications thereof as DOIsor FAs, applying the classifier to at least some of the potentialdefects to obtain an estimation of a number of expected DOIs in thespecimen.